Adaptive, self-tuning virtual sensing system for cyber-attack neutralization

ABSTRACT

An industrial asset may have a plurality of monitoring nodes, each monitoring node generating a series of monitoring node values over time representing current operation of the industrial asset. An abnormality detection computer may determine that an abnormal monitoring node is currently being attacked or experiencing a fault. An autonomous, resilient estimator may continuously execute an adaptive learning process to create or update virtual sensor models for that monitoring node. Responsive to an indication that a monitoring node is currently being attacked or experiencing a fault, a level of neutralization may be automatically determined. The autonomous, resilient estimator may then be dynamically reconfigured to estimate a series of virtual node values based on information from normal monitoring nodes, appropriate virtual sensor models, and the determined level of neutralization. The series of monitoring node values from the abnormal monitoring node or nodes may then be replaced with the virtual node values.

This invention was made with Government support under contract numberDE-OE0000903 awarded by the Department of Energy. The Government hascertain right in this invention.

BACKGROUND

Industrial control systems that operate physical systems (e.g.,associated with power turbines, jet engines, locomotives, autonomousvehicles, etc.) are increasingly connected to the Internet. As a result,these control systems have been increasingly vulnerable to threats, suchas cyber-attacks (e.g., associated with a computer virus, malicioussoftware, etc.), that could disrupt electric power generation anddistribution, damage engines, inflict vehicle malfunctions, etc. Currentmethods primarily consider attack detection in Information Technology(“IT,” such as, computers that store, retrieve, transmit, manipulatedata) and Operation Technology (“OT,” such as direct monitoring devicesand communication bus interfaces). Cyber-attacks can still penetratethrough these protection layers and reach the physical “domain” as seenin 2010 with the Stuxnet attack. Such attacks can diminish theperformance of a control system and may cause total shut down orcatastrophic damage to a plant. Currently, no methods are available toautomatically detect, during a cyber-incident, attacks at the domainlayer where sensors, controllers, and actuators are located. In somecases, multiple attacks may occur simultaneously (e.g., more than oneactuator, sensor, or parameter inside control system devices might bealtered maliciously by an unauthorized party at the same time). Notethat some subtle consequences of cyber-attacks, such as stealthy attacksoccurring at the domain layer, might not be readily detectable (e.g.,when only one monitoring node, such as a sensor node, is used in adetection algorithm). Existing approaches to protect an industrialcontrol system, such as failure and diagnostics technologies, may notadequately address these problems—especially when multiple, simultaneousattacks occur since such multiple faults/failure diagnostic technologiesare not designed for detecting stealthy attacks in an automatic manner.

It may be important to maintain an industrial asset's functionalityduring an attack. For example, an operator may want a power generationplant to continue to provide electricity even when one or more sensors,actuators, etc. are the subject of a cyber-attack. It may similarly bedesired to operate the asset when one or more monitoring nodes fail.Moreover, it may be advantageous to provide protection for an industrialasset without requiring redundant components (e.g., industrial controlsystems) and/or any major changes and/or re-design of controllers.

SUMMARY

According to some embodiments, an industrial asset may be associatedwith a plurality of monitoring nodes, each monitoring node generating aseries of monitoring node values over time that represent operation ofthe industrial asset. An abnormality detection computer may determinethat at least one abnormal monitoring node is currently being attackedor experiencing a fault. An autonomous, resilient estimator maycontinuously execute an adaptive learning process to create or updatevirtual sensor models for the monitoring nodes. Responsive to anindication that a monitoring node is currently being attacked orexperiencing a fault, a level of neutralization may be automaticallydetermined. The autonomous, resilient estimator may then be dynamicallyreconfigured to estimate a series of virtual node values for theabnormal monitoring node or nodes based on information from normalmonitoring nodes, appropriate virtual sensor models, and the determinedlevel of threat neutralization. The series of monitoring node valuesfrom the abnormal monitoring node or nodes may then be replaced with thevirtual node values.

Some embodiments comprise: means for determining, by an abnormalitydetection computer, that at least one abnormal monitoring node iscurrently being attacked or experiencing a fault; means for continuouslyexecuting, by an autonomous, resilient estimator, an adaptive learningprocess to create or update virtual sensor models for the monitoringnodes; responsive to an indication that at least one abnormal monitoringnode is currently being attacked or experiencing a fault, means forautomatically determining a level of threat neutralization; means fordynamically reconfiguring the autonomous, resilient estimator toestimate a series of virtual node values for the abnormal monitoringnode or nodes based on information from normal monitoring nodes,appropriate virtual sensor models, and the determined level ofneutralization; and means for replacing the series of monitoring nodevalues from the abnormal monitoring node or nodes with the virtual nodevalues.

Some technical advantages of some embodiments disclosed herein areimproved systems and methods to protect an industrial asset fromcyber-attacks in an automatic and accurate manner.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a high-level block diagram of a system to protect anindustrial asset according to some embodiments.

FIG. 2 is an industrial asset protection method in accordance with someembodiments.

FIG. 3 is a block diagram of an industrial asset protection systemaccording to some embodiment.

FIG. 4 illustrates a method of generating an abnormality alert inaccordance with some embodiments.

FIGS. 5 and 6 illustrate features, feature vectors, and decisionboundaries in accordance with some embodiments.

FIG. 7 is an abnormality detection model creation method according tosome embodiments.

FIG. 8 is a correlation heat map of monitoring nodes in accordance withsome embodiments.

FIG. 9 includes a portion of a virtual sensor lookup table according tosome embodiments.

FIG. 10 is an example of a global threat protection system in accordancewith some embodiments when multiple gas turbines are involved in asystem.

FIG. 11 is a high-level block diagram of a system to protect anindustrial asset according to some embodiments.

FIG. 12 is a more detailed method of protecting an industrial assetaccording to some embodiments.

FIG. 13 illustrates an anti jitter process in accordance with someembodiments.

FIG. 14 is a more detailed architecture of a system to protect anindustrial asset according to some embodiments.

FIG. 15 is a method that might be associated with an on-line operationalprocess in accordance with some embodiments.

FIG. 16 is a method of determining whether an attack is an independentattack or a dependent attack according to some embodiments.

FIG. 17 illustrates a feature time series of an attack comparing thereal-time feature of a monitoring node to the modeled feature of themonitoring node according to some embodiments.

FIG. 18 illustrates a feature time series of a stealthy attack comparingthe real-time feature of a monitoring node to the modeled feature of amonitoring node in accordance with some embodiments.

FIG. 19 is an example of attack localization in a multiple-attackscenario according to some embodiments.

FIG. 20 is a causal dependency matrix of monitoring nodes in accordancewith some embodiments.

FIG. 21 is an autonomous reconfigurable virtual sensing systemarchitecture according to some embodiments.

FIG. 22 illustrates a sliding window technique for real-timemeasurements in accordance with some embodiments.

FIG. 23 is a block diagram of an industrial asset protection platformaccording to some embodiments of the present invention.

FIG. 24 is a tabular portion of a virtual sensor database in accordancewith some embodiments.

FIG. 25 is a virtual sensor display according to some embodiments.

FIG. 26 is an autonomous reconfigurable virtual sensing system inaccordance with some embodiments.

FIG. 27 is an adaptive, self-tuning neutralization system according tosome embodiments.

FIG. 28 is logic for switching from virtual sensors back to physicalsensors in accordance with some embodiments.

DETAILED DESCRIPTION

In the following detailed description, numerous specific details are setforth in order to provide a thorough understanding of embodiments.However, it will be understood by those of ordinary skill in the artthat the embodiments may be practiced without these specific details. Inother instances, well-known methods, procedures, components and circuitshave not been described in detail so as not to obscure the embodiments.

Industrial control systems that operate physical systems areincreasingly connected to the Internet. Note that, as used herein, theterm “industrial” might be associated with any system that is connectedto an external source, such as the Internet in the case of acyber-physical system or locally operating an air-gapped physicalsystem. As a result, these control systems have been increasinglyvulnerable to threats and, in some cases, multiple attacks may occursimultaneously. Protecting an asset may depend on detecting such attacksas well as naturally occurring faults and failures. Existing approachesto protect an industrial control system, such as failure and diagnosticstechnologies, may not adequately address these threats—especially whenmultiple, simultaneous attacks occur. It would therefore be desirable toprotect an industrial asset from cyber threats in an automatic andaccurate manner. In particular, an operator of an industrial asset mightwant to implement “accommodation” procedures such that criticalfunctions of the asset may automatically still function even in theevent of one or more cyber-attacks or monitoring node failure (e.g., byreplacing unhealthy sensor node data values with virtual sensor datavalues based on information obtained from other, healthy nodes). FIG. 1is a high-level architecture of a system 100 that might be used toprotect an industrial asset such as a gas turbine. The system 100 mayinclude a plurality of monitoring nodes 110, each monitoring nodegenerating a series of monitoring node values over time that representoperation of the industrial asset (e.g., a temperature, a speed, avoltage, etc.). An abnormality detection computer 160 coupled to themonitoring nodes 110 may be adapted to determine that a particularmonitoring node is currently being attacked by a cyber-threat or isexperiencing a failure (e.g., a sensor might be stuck). An autonomous,resilient estimator 150 may receive an indication of the abnormalmonitoring node and, as a result, estimate a series of virtual nodevalues for the attacked monitoring node based on information receivedfrom monitoring nodes that are not currently being attacked (e.g., usinga lookup table 155). In some embodiments, an estimation of series ofvirtual node values happens in real-time during normal operation asopposed to estimating the virtual node values after the abnormalmonitoring node information is received. Soon after the abnormalmonitoring node information is received, signals from abnormalmonitoring nodes are replaced by the most current virtual node values.The virtual sensor may then replace the series of monitoring node valuesfrom the attacked monitoring node with the virtual node values (e.g., asillustrated by the dashed arrow output 152 in FIG. 1).

FIG. 2 is an industrial asset protection method that might be associatedwith the elements of the system of FIG. 1. Note that the flowchartsdescribed herein do not imply a fixed order to the steps, andembodiments of the present invention may be practiced in any order thatis practicable. Note that any of the methods described herein may beperformed by hardware, software, or any combination of these approaches.For example, a computer-readable storage medium may store thereoninstructions that when executed by a machine result in performanceaccording to any of the embodiments described herein.

At S210, they system may continuously execute (e.g., on-line as theindustrial asset operates) an adaptive learning process to create (whenno pre-determined model is available) or update (when a pre-determinedmodel was created off-line) virtual sensing models in accordance withany of the embodiments described herein. The adaptive learning processmight be associated with, for example, a reinforcement learning method.

At S220, an abnormality detection computer might determine that one ormore monitoring nodes is currently abnormal (e.g., the node beingattacked and/or is experiencing a failure). At S230, an appropriatelevel of neutralization is determined (e.g., as described in connectionwith FIG. 27). At S240, an autonomous, resilient estimator may estimatea series of virtual node values for the abnormal monitoring node basedon the determined level of neutralization and information received frommonitoring nodes that are currently normal. That is, information from“healthy” monitoring nodes may be used to estimate data from amonitoring node that is behaving abnormally. Note that the estimationsassociated with S240 might be performed even before a determination ismade that a particular node is currently being attacked (e.g., at S220).At S250, the series of monitoring node values from the attackedmonitoring node may be replaced with the virtual node values: As aresult, the industrial asset may continue to operate even whenundergoing a cyber-attack or monitoring node failure. According to someembodiments, the series of virtual node values are estimated directly intime space (e.g., after pre-filtering as described herein).

Note that a determination that a particular monitoring node is currentlyabnormal might be based on an abnormality detection model created forthe industrial asset. For example, FIG. 3 is an example of an industrialasset protection system 300. The system 300 may include a “normal space”data source 320 storing, for each of a plurality of monitoring nodes310, a series of normal values over time that represent normal operationof an industrial asset (e.g., collected from actual monitoring node 310data as illustrated by the dashed line in FIG. 3). The system 300 mayalso include an “abnormal space” data source 330 storing series ofvalues over time associated with monitoring nodes undergoing acyber-attack (e.g., as recorded during an actual attack or as predictedby a high-fidelity physics-based industrial asset model) and/orexperiencing a failure.

Information from the normal space data source 320 and the abnormal spacedata source 330 may be provided to an abnormality detection modelcreation computer 360 that uses this data to create a decision boundary(that is, a boundary that separates normal behavior from abnormalbehavior). The decision boundary may then be used by an abnormalitydetection computer 350 executing an abnormality detection model 355. Theabnormality detection model 355 may, for example, monitor streams ofdata from the monitoring nodes 310 comprising data from sensor nodes,actuator nodes, and/or any other critical monitoring nodes (e.g.,monitoring nodes MN₁ through MN_(N)) and automatically output anabnormality alert (e.g., indicating that various monitoring nodes of theindustrial asset are normal, attacked, or experiencing a fault) to oneor more remote monitoring devices 370 when appropriate (e.g., fordisplay to a user) and/or to an autonomous, resilient estimator. As usedherein, the terms “automatically” or “autonomous” may refer to, forexample, actions that can be performed with little or no humanintervention. According to some embodiments, information about adetected abnormality may also be transmitted back to an industrialcontrol system.

As used herein, devices, including those associated with the system 300and any other device described herein, may exchange information via anycommunication network which may be one or more of a Local Area Network(“LAN”), a Metropolitan Area Network (“MAN”), a Wide Area Network(“WAN”), a proprietary network, a Public Switched Telephone Network(“PSTN”), a Wireless Application Protocol (“WAP”) network, a Bluetoothnetwork, a wireless LAN network, and/or an Internet Protocol (“IP”)network such as the Internet, an intranet, or an extranet. Note that anydevices described herein may communicate via one or more suchcommunication networks.

The abnormality detection model creation computer 360 may storeinformation into and/or retrieve information from various data stores,such as the normal space data source 320 and the abnormal space datasource 330. The various data sources may be locally stored or resideremote from the abnormality detection model creation computer 360.Although an abnormality threat detection model creation computer 360 isshown in FIG. 3, any number of such devices may be included. Moreover,various devices described herein might be combined according toembodiments of the present invention. For example, in some embodiments,the abnormality detection model creation computer 360, normal space datasource 320, and abnormal space data source 330 might comprise a singleapparatus. The abnormality detection model creation computer 360 and/orabnormality detection computer 350 functions may be performed by aconstellation of networked apparatuses, in a distributed processing orcloud-based architecture.

A user may access the system 300 via one of the monitoring devices 370(e.g., a Personal Computer (“PC”), tablet, or smartphone) to viewinformation about and/or manage attack and fault information inaccordance with any of the embodiments described herein. In some cases,an interactive graphical display interface may let a user define and/oradjust certain parameters (e.g., attack detection trigger levels ormodel configurations) and/or provide or receive automatically generatedrecommendations or results from the abnormality detection model creationcomputer 360 and/or the abnormality detection computer 350.

The decision boundary associated with the abnormality detection model355 can be used to detect cyber-attacks. For example, FIG. 4 is anindustrial asset protection method that might be implemented accordingto some embodiments. At S410, the system may receive, from a pluralityof monitoring nodes, a series of current values over time that representa current operation of an industrial asset. The system may alsogenerate, based on the received series of current values, a set ofcurrent feature vectors. At S420, an abnormality detection model may beaccessed including at least one decision boundary. At S430, the modelmay be executed and an abnormality alert may be transmitted (e.g., to anautonomous, resilient estimator) based on the set of current featurevectors and the decision boundary when appropriate (e.g., when acyber-attack or fault is detected). According to some embodiments, oneor more response actions may be performed when an abnormality alert istransmitted. For example, the system might automatically shut down allor a portion of the industrial asset (e.g., to let the detectedpotential cyber-attack or fault be further investigated). As otherexamples, one or more parameters might be automatically modified, asoftware application might be automatically triggered to capture dataand/or isolate possible causes, a virtual sensor might be created ordeployed, etc.

When available, a system may take advantage of the physics of anindustrial asset by learning a priori from tuned high fidelity equipmentmodels and/or actual “on the job” data to detect single or multiplesimultaneous adversarial threats to or faults in the system. Moreover,monitoring node data may be converted to features using advancedfeature-based methods, and the real-time operation of the control systemmay be monitoring in substantially real-time. Abnormalities may bedetected by classifying the monitored data as being “normal” or“abnormal” (e.g., “attacked”). This decision boundary may be constructedin feature space using dynamic models and may help enable earlydetection of vulnerabilities (and potentially avert catastrophicfailures) allowing an operator to restore the control system to normaloperation in a timely fashion. Note, however, that in many cases aphysics-based model of an industrial asset might not be readilyavailable.

FIGS. 5 and 6 illustrate features, feature vectors, and decisionboundaries in accordance with some embodiments. In particular, FIG. 5illustrates 500 boundaries and feature vectors for a monitoring nodeparameter in accordance with some embodiments. A graph 510 includes afirst axis representing value weight 1 (“w1”), a feature 1, and a secondaxis representing value weight 2 (“w2”), a feature 2. Values for w1 andw2 might be associated with, for example, outputs from a PrincipalComponent Analysis (“PCA”) performed on input data. PCA might be one ofthe features that might be used by the algorithm to characterize thedata, but note that other features could be leveraged. The graph 510illustrated in FIG. 5 represents compressor discharge temperature for agas turbine but other values might be monitored instead (e.g.,compressor pressure ratio, compressor inlet temperature, fuel flow,generator power, gas turbine exhaust temperature, etc.). The graph 510includes an average boundary 512 (solid line), a minimum boundary 514(dotted line), a maximum boundary 516 (dashed line), and an indicationassociated with current feature location for the monitoring nodeparameter (illustrated with an “X” on the graph 510). As illustrated inFIG. 5, the current monitoring node location is between the minimum andmaximum boundaries (that is, the “X” is between the dotted and dashedlines). As a result, the system may determine that the operation of theindustrial asset is normal (and no attack or fault is being detected forthat monitoring node). FIG. 6 illustrates 600 three dimensions of threatnode outputs in accordance with some embodiments. In particular, a graph610 plots monitoring node outputs during normal operation (“+”) and whenunder attack or experiencing a fault (“−”) in three dimensions, such asdimensions associated with PCA: w1, w2, and w3. Moreover, the graph 610includes a dashed line indication of a normal operating space decisionboundary 620.

Note that an appropriate set of multi-dimensional feature vectors, whichmay be extracted automatically (e.g., via an algorithm) and/or bemanually input, might comprise a good predictor of measured data in alow dimensional vector space. According to some embodiments, appropriatedecision boundaries may be constructed in a multi-dimensional spaceusing a data set which is obtained via scientific principles associatedwith Design of Experiments (“DoE”) techniques. Moreover, multiplealgorithmic methods (e.g., support vector machines or other machinelearning based supervised learning techniques) may be used to generatedecision boundaries. Since boundaries may be driven by measured data,defined boundary margins may help to create a threat zone in amulti-dimensional feature space. Moreover, the margins may be dynamic innature and adapted based on a transient or steady state model of theequipment and/or be obtained while operating the system as inself-learning systems from incoming data stream. According to someembodiments, a training method may be used for supervised learning toteach decision boundaries. This type of supervised learning may takeinto account an operator's knowledge about system operation (e.g., thedifferences between normal and abnormal operation).

FIG. 7 illustrates a model creation method that might be performed bysome or all of the elements of the system 100, 300 described withrespect to FIGS. 1 and 3. At S710, the system may receive, for each of aplurality of monitoring nodes, a series of normal values over time thatrepresent normal operation of the industrial asset and a set of normalfeature vectors may be generated. At S720, the system may retrieve, foreach of the plurality of monitoring nodes, a series of abnormal valuesover time that represent abnormal operation of the industrial asset anda set of abnormal feature vectors may be generated. The series of normalvalues might be obtained, for example, by DoE on an industrial controlsystem associated with a power turbine, a jet engine, a locomotive, anautonomous vehicle, etc. At S730, a decision boundary may beautomatically calculated and output for an abnormality detection modelbased on the sets of normal and abnormal feature vectors. According tosome embodiments, the decision boundary might be associated with a line,a hyperplane, a non-linear boundary separating normal space fromattacked space, and/or a plurality of decision boundaries. In addition,note that the abnormality detection model might be associated with thedecision boundary, feature mapping functions, and/or feature parameters.

Thus, a system may classify the status of an industrial control systemhaving a plurality of monitoring nodes (including sensor, actuator, andcontroller nodes) as being normal or abnormal. This may enable tailored,resilient, and fault-tolerant control remedies, including the deploymentof virtual sensors, against cyber-attacks and faults.

According to some embodiments, time-series data may be received from acollection of monitoring nodes (e.g., sensor, actuator, and/orcontroller nodes). Features may then be extracted from the time seriesdata for each monitoring node. The term “feature” may refer to, forexample, mathematical characterizations of data. Examples of features asapplied to data might include the maximum and minimum, mean, standarddeviation, variance, settling time, Fast Fourier Transform (“FFT”)spectral components, linear and non-linear principal components,independent components, sparse coding, deep learning, etc. The type andnumber of features for each monitoring node, might be optimized usingdomain-knowledge, feature engineering, or ROC statistics. The localfeatures for each monitoring node may be stacked to create the globalfeature vector. The global feature vector may also contain interactivefeature involving two or more monitoring nodes, e.g. cross-correlationbetween two nodes. According to some embodiments, the features may benormalized and the dimension of the global feature vector can then befurther reduced using any dimensionality reduction technique such asPCA. Note that the features may be calculated over a sliding window ofthe signal time series and the length of the window (and the duration ofthe slide) may be determined from domain knowledge and inspection of thedata or using batch processing.

Note that many different types of features may be utilized in accordancewith any of the embodiments described herein, including principalcomponents (weights constructed with natural basis sets) and statisticalfeatures (e.g., mean, variance, skewness, kurtosis, maximum, minimumvalues of time series signals, location of maximum and minimum values,independent components, etc.). Other examples include deep learningfeatures (e.g., generated by mining experimental and/or historical datasets) and frequency domain features (e.g., associated with coefficientsof Fourier or wavelet transforms). Embodiments may also be associatedwith time series analysis features, such as cross-correlations,auto-correlations, orders of the autoregressive, moving average model,parameters of the model, derivatives and integrals of signals, risetime, settling time, neural networks, etc. Still other examples includelogical features (with semantic abstractions such as “yes” and “no”),geographic/position locations, and interaction features (mathematicalcombinations of signals from multiple monitoring nodes and specificlocations). Embodiments may incorporate any number of features, withmore features allowing the approach to become more accurate as thesystem learns more about the physical process and threat. According tosome embodiments, dissimilar values from monitoring nodes may benormalized to unit-less space, which may allow for a simple way tocompare outputs and strength of outputs.

Note that PCA information may be represented as weights in reduceddimensions. For example, data from each monitoring node may be convertedto low dimensional features (e.g., weights). According to someembodiments, monitoring node data is normalized as follows:

${S_{normalized}(k)} = \frac{{S_{nominal}(k)} - {S_{original}(k)}}{{\overset{\_}{S}}_{nominal}}$where S stands for a monitoring node quantity at “k” instant of time.Moreover, the output may then be expressed as a weighted linearcombination of basis functions as follows:

$S = {S_{0} + {\sum\limits_{j = 1}^{N}{w_{i}\Psi_{j}}}}$where S₀ is the average monitoring node output with all threats, w_(j)is the j^(th) weight, and Ψ_(j) is the j^(th) basis vector. According tosome embodiments, natural basis vectors are obtained using a covarianceof the monitoring nodes' data matrix. Once the basis vectors are known,the weight may be found using the following equation (assuming that thebasis sets are orthogonal):w _(j)=(S−S ₀)^(T)Ψ_(j)Note that weights may be an example of features used in a featurevector.

Thus, once the observed quantities from monitoring nodes are expressedin terms of feature vectors (e.g., with many features), the featurevectors may then be used as points in a multi-dimensional feature space.During real-time abnormality detection, decisions may be made bycomparing where each point falls with respect to a decision boundarythat separates the space between two regions (or spaces): abnormal(“attack” or “fault”) space and normal operating space. If the pointfalls in the abnormal space, the industrial asset is undergoing anabnormal operation such as during a cyber-attack. If the point falls inthe normal operating space, the industrial asset is not undergoing anabnormal operation such as during a cyber-attack or fault. In someembodiments, an appropriate decision zone with boundaries is constructedusing data sets as described herein with high fidelity models. Forexample, support vector machines may be used with a kernel function toconstruct a decision boundary. According to some embodiments, deeplearning techniques may be used to construct decision boundaries.

Note that industrial processes may be controlled by Programmable LogicControllers (“PLC”) with Ethernet ports and IP addresses. Computer wormscan live in the PLC and be inactive for many days and can replicateitself into many targets as it finds them. IT and OT protectionmechanisms cannot completely keep a PLC safe and different approachesmay be needed to protect critical infrastructures from more advancedviruses and allow for an industrial asset to operate (including criticalfunctions) even when being attacked. In particular some embodimentsdescribed herein provide a multi-node virtual sensor to sustainoperation of an industrial asset with no loss of critical function. Thevirtual sensor might utilize, for example, some or all of the followinginformation to estimate true signals; (1) information from localizationabout which nodes were attacked independently, (2) features frommonitoring nodes, and (3) a multi-node feature-based virtual sensormodel trained a priori from the system data set. Estimated true signalsmay then be used in the respective nodes instead of attacked signals.

In a control system during operational normalcy, the system may receivetime series signals from various monitoring nodes (i.e., sensor,actuator, controller, etc.). Consider a general system (e.g., cyberphysical system, software system, bio-mechanical system, network system,communication system, etc.) that contains access to continuous streamsof data in the form of time series signals from all these sensors. Thetime series signals might be generated from a set of output sensor nodes(“y”; both physical and virtual sensors already incorporated in thesystem), a set of actuator nodes (“u”; both hard and soft actuatorsgenerated from open or closed loop system), a set of output ofcontroller nodes (“c”; controller node signals), and a set of referencenodes (“r”; reference signals). According to some embodiments, logicalsare also considered as time series signals. Some or all combinations ofthese signals may be used for the purpose of accommodation with avirtual sensor. The virtual sensor matrix used for this purpose may, forexample, estimate not only system sensor outputs, y, when an attacktakes place to any of the sensor nodes, but also other signals to thecontrol system; actuator node signals, u, controller node signals, c;reference signals, r, etc. Thus, the virtual sensor-based accommodationsystem may provide an intelligent system that is designed to estimatesignals that are corrupted/attacked from the healthy signals itreceives.

Some embodiments described herein may provide a system and method forautonomous reconfigurable virtual sensing to neutralize the effect ofanomalies (cyber-attack or faults) in system measurements. The systemmay provide correct estimates of compromised sensor measurements usinguncompromised sensor measurements, thus replacing the comprised sensorswith healthy virtual (or “soft”) sensors. The autonomous, resilientestimator may use, according to some embodiments, continuous adaptivelearning. For example, virtual sensor estimations may be computedon-line (during operation of the industrial asset) using an adaptiverecursive method based on reinforcement learning. The system may bescalable, efficient, and automatically adjust its configuration toaccommodate the time-varying uncompromised portion of the systemsensors. Note that the system might work with partial, or no, a prioriknowledge (e.g., a predetermined virtual sensor model).

Some embodiments described herein may provide a resilient estimationmethod for sensors of a control system to maintain the integrity andavailability of the system under abnormalities such as cyber-attacks andsensor faults/failures. According to some embodiments, a virtual sensingsystem may satisfy some or all of the following four criteria:

-   -   1. the virtual estimator is unbiased (i.e., zero-mean error);    -   2. the virtual estimator has white innovation (optimal in the        sense of a Cramer-Rao information bound);    -   3. the virtual estimator is statistically efficient (i.e., the        error asymptotically converging to zero); and    -   4. the estimation error standard division is comparable to the        real sensor measurement (so the quality of the virtual        estimations is comparable with the physical sensor measurement).

Note that a system may receive time-series data from a collection ofsensor monitoring nodes and replace independently attacked/faultysensor(s) with their virtual estimate(s) as soon as an abnormality isdetected. For each compromised sensor, the system may construct anautonomous, resilient estimator using uncompromised sensors. Each ofsuch autonomous, resilient estimators can use all (or a subset) of theremaining healthy sensors. For example, for each sensor, an Analysis OfVariance (“ANOVA”) or correlation/regression analysis may be performedto rank the contributing factors. The system may then down-select thesignificant sensors, which are desirable for virtual modeling of eachparticular on-line sensor estimator. Then, using the aforementionedANOVA or correlation analysis, the list of the factors to be used ineach virtual model may pre-stored into the system, while the virtualsensing model is learnt and adapted online.

For example, FIG. 8 shows a correlation heat map 800 for ten monitoringnodes (sensors/actuators/controller nodes) of a gas turbine. Pairs ofvalues may each have a correlation scores (e.g., from 1 indicating astrong correlation to zero indicating no correlation to −1 indicating astrong negative correlation). For each node, the other nodes whoseabsolute value of the correlation coefficient is larger than a threshold(e.g., above 0.25) might be stored as main contributing factors. For thecritical sensors of the system, or the ones that measure highlynonlinear dynamic phenomena, an off-line model could be learned and thenadapted online. For the rest of the sensors, the on-line learning maystart completely model-free and the models may be learned from scratch,in real-time during operation of the industrial asset. Note that avirtual sensor may utilize a lookup table, such as the table 900illustrated in FIG. 9 including a virtual sensor matrix 910 and a signalbeing estimated 920, to create a value Y_(i)=C_(i,jXi) (where irepresents the signals in feature space being estimated and j representsthe number of attacked signals being estimated).

Some embodiments described herein may assume that when theattacked/faulty sensors are removed, the compromised plant remainsobservable. The continuous learning may be based on ReinforcementLearning (“RL”) methodology. For example, an online learning algorithmsuch as Q-learning or the recursive least-squares method might be usedfor reinforcement learning. According to some embodiments, the approachmight be interpreted as a Partially Observed Markov Decision Process(“POMDP”) with continuous state and action spaces. This POMDP mayexhibit, for example, deterministic transitions when configurationtransitions are specified by a sensor diagnostics and anomalyclassification module. A reinforcement learning engine can work on adeep neural network using Q-learning thus comprising a deep Q-network.

During normal operation, all sensors go into a reinforcement learningmethod running an online learning algorithm (e.g., a recursiveleast-square, a recursive weighted least square, Q-learning, etc.). Thismay comprise a “base” configuration of the system. The baseconfiguration remains in place as long as there are no reportedabnormalities (i.e., attacks or faults). Once an abnormality isreported, the virtual sensing system automatically adopts into a“partial” configuration for which the healthy sensors are the inputs andthe estimates of both the compromised sensors as well normal sensors arethe outputs. According to some embodiments, the system may keep thehealthy sensors in the estimation loop (I.e., forming a full-orderobserver) so that at each instant a learnt model for virtual estimationsof all sensors is readily available. In this way, if another sensor issuddenly compromised, the system will keep running without facingdiscontinuity in the underlying optimization procedures of continuouslearning. Inside the partial configuration, all or a subset of inputsmay be used to compute each particular output. The virtual sensorestimator may be a full-order observer both during the base and partialconfigurations, hence providing estimates of the measurements of thesensors at all times. The correlation analysis previously describe maybe used to provide initial guess for the reward/penalty weightingfunctions in the reinforcement learning.

The continuous learning described herein may serve as a core of amodel-free (or partial-model) Kalman filter, which receives partial orfull measurements (depending of the status of the system) and providefull-order (or reduced-order) output estimates. A Kalman TemporalDifferences technique may be used to implement the Kalman Filter. Thedescribed reinforcement learning based continuous learning framework maysatisfies the conditions 1 through 4 previously mentioned as long as theplant remains observable through usage of the uncompromised subset ofsensors. If the plant loses this observability due to large number ofsensors being compromised, the system may still provide virtual sensorestimates but some or all of the conditions may no longer be satisfied.According to some embodiments, an online observability test may beperformed using the models built online and a warning may be generatedby the autonomous, resilient estimator in this situation. In addition,statistical tests (such as X² test) may be performed online using theinnovation signal of the uncompromised sensor measurements, which arereadily available verses their virtual estimates, which are part of thevirtual estimator outputs.

Note that feature vectors might represent local or global information.For example, FIG. 10 is an example of a global threat protection system1000 in accordance with some embodiments when multiple gas turbines areinvolved in a system. In particular, the system 1000 includes threeturbines (A, B, and C) and batches of values 1010 from threat nodes arecollected for each generated over a period of time (e.g., 60 to 80seconds). According to some embodiments, the batches of values 1010 fromthreat nodes overlap in time. The values 1010 from threat nodes may, forexample, be stored in a matrix 1020 arranged by time (t₁, t₂, etc.) andby type of threat node (S₁, S₅, etc.). Feature engineering components1030 may use information in each matrix 1020 to create a feature vector1040 for each of the three turbines (e.g., the feature vector 1040 forturbine C might include FS_(C1), FS_(C2), etc.). The three featurevectors 1040 may then be combined into a single global feature vector1050 for the system 1000. Interaction features 1060 may be applied(e.g., associated with A*B*C, A+B+C, etc.) and an anomaly detectionengine 1070 may compare the result with a decision boundary and output athreat alert signal when appropriate.

Some embodiments described herein are directed to challenging problemsencountered during rapid transients associated with an industrial asset(e.g., in a rapidly changing environment). For example, FIG. 11 is ahigh-level block diagram of a system 1100 to protect an industrial assetaccording to some embodiments. The system 1100 extracts local featuresfor monitoring nodes 1-N 1120, 1122 from time-domain values 1110. Thesystem 1100 then smooths local transient capturing features for eachnode 1130, 1132 (e.g., via band-pass filters). The system 1100 may also,according to some embodiments, extract global features 1140 and computea decision boundary score 1160. The decision boundary score 1160 canthen be compared to a threshold value 1170 to determine if the newsystem status of the industrial asset is “attack” or “normal.”

FIG. 12 is a more detailed method of protecting an industrial assetaccording to some embodiments. At S1210, the system may evaluatetransients at the monitoring node level (called ‘features for capturinglocal transients’). At S1220, the system may evaluate transients atasset level (called “features for capturing global transients”). AtS1230, smoothing filters may be applied for transient capturing featuretime series data. At S1240, anti-jittering (e.g., robustification)methods may be employed in accordance with any of the embodimentsdescribed herein.

With respect to key features for capturing local transients, someembodiments may be associated with local transient capturing features oftime-domain node values. For example, the local feature vector for eachmonitoring node might comprise of a set of “base features,” which may beof any of the types described herein, augmented by a set of “transientcapturing features” that provide, in rapid scale, a sense for the notionof change in the environment. In other words, the transient capturingfeatures may characterize how fast the environment is evolving. In thefollowing, a few types of transient capturing feature types will bedescribed to enable such a distinguishing nature of rapid changes. Thefollowing description is for illustration purposes only and thetransient capturing features are not limited to these specific types.

Some local transients capturing features may be associated withtime-derivative local features of the time-domain values. That is, oneparticular type of transient capturing features might be the timederivate(s) of the monitoring node values. To this end, numericaldifferentiation can be used with central or backward finite differences.The numerical derivatives may be computed using the last points of thebatched data. The number of points needed might be selected based on theorder and type of the differentiation method. For example, the first andsecond derivatives using backward finite difference for monitoring nodex₁ are:

${{\frac{{dx}_{1}}{dt}(t)} = \frac{{x_{1}(t)} - {x_{1}( {t - T_{s}} )}}{T_{s}}}{{\frac{d^{2}x_{1}}{{dt}^{2}}(t)} = \frac{{x_{1}(t)} - {2{x_{2}( {t - T_{s}} )}} + {x_{1}( {t - {2T_{s}}} )}}{T_{s}^{2}}}$where T_(s) is the sampling time. Higher-order derivatives are alsopossible to compute, but for practical reasons (and depending on hownoisy the data is) numerical differentiation beyond first and secondderivatives may be impractical. The above computations are simple andlow cost, and no additional storage might be required since a batch oftime-series data (usually of the length of several T_(s)) may enter thesystem as each sampling time.

Transit capturing features can also be extracted as the features of thebase features, capturing the evolution of the base features for eachmonitoring node. These are essentially “features of features.” Forexample, time-derivative features of the local base features might beemployed. These may be similar to the time-derivative features of themonitoring node values described above, and time-domain derivatives ofthe base features may also be computed from feature evolutiontime-series data. For example, if w₁ ¹ is the first local base featureof the first monitoring node x₁, then:

${{\frac{{dw}_{1}^{1}}{dt}(t)} = \frac{{w_{1}^{1}(t)} - {w_{1}^{1}( {t - T_{s}} )}}{T_{s}}}{{\frac{d^{2}w_{1}^{1}}{{dt}^{2}}(t)} = \frac{{w_{1}^{1}(t)} - {2{w_{1}^{1}( {t - T_{s}} )}} + {w_{1}^{1}( {t - {2T_{s}}} )}}{T_{s}^{2}}}$Note that, similar to the time-domain values of the monitoring nodes,after the base features are extracted, time-domain values of basefeatures may also be readily available for each sliding batch. Also notethat the sampling time, T_(s), may not be same as the local features oftime-domain data. Here again, the computations may be low-cost butadditional storage may be needed since they involve features computedfrom previous batches of data.

Other embodiments might be associated with local “Jacobian features.”Local Jacobian features are the partial derivatives of a base feature ofa monitoring node with respect to another base feature of the samemonitoring node. Suppose the local base feature vectors of an examplemonitoring node x₁ are W¹=[w₁ ¹ . . . w_(ƒ) ₁ ¹]^(T), where ƒ₁ is thenumber of local base features of the monitoring node x₁. The Jacobian ofvector field W¹ with respect to itself, is a matrix with the maindiagonal elements being 1:

${J_{W^{1}}( W^{1} )} = \begin{bmatrix}1 & \frac{\partial w_{1}^{1}}{\partial w_{2}^{1}} & \cdots & \frac{\partial w_{1}^{1}}{\partial w_{f_{1}}^{1}} \\\frac{\partial w_{2}^{1}}{\partial w_{1}^{1}} & 1 & \cdots & \frac{\partial w_{2}^{1}}{\partial w_{f_{1}}^{1}} \\ \vdots & \vdots & \ddots & \vdots \\\frac{\partial w_{f_{1}}^{1}}{\partial w_{1}^{1}} & \frac{\partial w_{f_{1}}^{1}}{\partial w_{2}^{1}} & \cdots & 1\end{bmatrix}$In this case, the local Jacobian features of x₁ are the non-diagonalelements of this matrix. Additionally, the properties of this matrixsuch as its trace, determinant, eigenvalues, etc. can be considered aslocal transit capturing features. The partial derivatives can benumerically computed using central or backward finite differences. Forexample, using backward finite differences:

${\frac{\partial w_{1}^{1}}{\partial w_{2}^{1}}(t)} = \frac{{w_{1}^{1}(t)} - {w_{1}^{1}( {t - T_{s}} )}}{{w_{2}^{1}(t)} - {w_{2}^{1}( {t - T_{s}} )}}$The local Jacobian features of the base features of other monitoringnodes may be computed in a similar fashion. Having n monitoring nodeseach having ƒ_(i), i=1, . . . , n local base features, in general, thereexists n of such Jacobian matrices, each having ƒ_(i)*(ƒ_(i)−1)time-varying elements.

Still other embodiments may be associated with local Hessian features.The Hessian of a vector field with respect to another vector field (oritself) is a tensor of order three, which can be shown as an array ofHessian matrices of each element of the vector field with respect to theother (or the same) vector field. In this case, for example, the Hessianof a vector field W¹ (local base features of monitoring node x₁) withrespect to itself is:H _(W) ₁ (W ¹)={H _(W) ₁ (w ₁ ¹),H _(W) ₁ (w ₂ ¹), . . . ,H _(W) ₁ (w_(ƒ) ₁ ¹)}For instance, H_(W) ₁ (w₁ ¹) is the Hessian of the first local basefeature w₁ ¹ with respect to the local base feature vector W¹, which is:

${H_{W^{1}}( w_{1}^{1} )} = \begin{bmatrix}0 & 0 & \cdots & 0 \\0 & \frac{\partial^{2}w_{1}^{1}}{\partial( w_{2}^{1} )^{2}} & \cdots & \frac{\partial^{2}w_{1}^{1}}{{\partial w_{2}^{1}}w_{f_{1}}^{1}} \\ \vdots & \vdots & \ddots & \vdots \\0 & \frac{\partial^{2}w_{1}^{1}}{{\partial w_{f_{1}}^{1}}w_{2}^{1}} & \cdots & \frac{\partial^{2}w_{1}^{1}}{\partial( w_{f_{1}}^{1} )^{2}}\end{bmatrix}$Note that the first row and first column of this matrix are zero. Formonitoring node x₁ with ƒ₁ local base feature, there are ƒ₁ such Hessianmatrices. The local Hessian features of node x₁ are the non-zeroelements of those matrices. The second partial derivate can also becomputed numerically using central or backward finite differences. Moreefficiently, the Hessian matrix may be computed using Hessian AutomaticDifferentiation (“HAD”).

The local features for each monitoring node (e.g., after de-noising) maybe stacked to create a global feature vector. The global feature vectormay also contain interactive feature involving two or more monitoringnodes, e.g. cross-correlation between two nodes. According to someembodiments, the features may be normalized. The dimension of the globalfeature vector can then be further reduced using any dimensionalityreduction technique such as PCA. Note that the transient capturingfeatures may be left out of such dimensionality reduction. The featuresmay be calculated over a sliding window of the signal time-series. Thelength of the window and the duration of slide might be determined, forexample, from domain knowledge and inspection of the data, detectionperformance, and computing resources. The interactive global featuresmay also contain global transit capturing features involving two or morenodes.

Global transient capturing features are essentially the partialderivatives of a time-domain values of a monitoring node with respect toother monitoring nodes, or the partial derivatives of the local featuresof a monitoring node with respect to the local features of othermonitoring nodes. According to some embodiments, global transientcapturing features of time-domain node values may be utilized, such as aJacobian feature of the time-domain values of the monitoring nodes.These features may be computed as the first partial derivatives oftime-domain values of a monitoring node with respect to another node.For example, suppose there are n monitoring nodes, X=[x₁, x₂, . . . ,x_(n)]^(T). The Jacobian of vector field X with respect to itself is:

${J_{X}(X)} = \begin{bmatrix}1 & \frac{\partial x_{1}}{\partial x_{2}} & \cdots & \frac{\partial x_{1}}{\partial x_{n}} \\\frac{\partial x_{2}}{\partial x_{1}} & 1 & \cdots & \frac{\partial x_{2}}{\partial x_{n}} \\ \vdots & \vdots & \ddots & \vdots \\\frac{\partial x_{n}}{\partial x_{1}} & \frac{\partial x_{n}}{\partial x_{2}} & \cdots & 1\end{bmatrix}$The off-diagonal elements can then be considered as global transitcapturing features. The partial derivatives can be computed usingcentral or backward finite differences. For example, using backwardfinite differences:

${\frac{\partial x_{1}}{\partial x_{2}}(t)} = \frac{{x_{1}(t)} - {x_{1}( {t - T_{s}} )}}{{x_{2}(t)} - {x_{2}( {t - T_{s}} )}}$

Other embodiments may use a Hessian feature of the time-domain values ofthe monitoring nodes. This is the Hessian of the vector field X withrespect to itself a tensor of order three which is represented by anarray of Hessian matrices, similar to the local Hessian features:H _(X)(X)={H _(X)(x ₁),H _(X)(x ₂), . . . ,H _(X)(x _(n))}For example, for H_(x)(x₁):

${H_{X}( x_{1} )} = \begin{bmatrix}0 & 0 & \cdots & 0 \\0 & \frac{\partial^{2}x_{1}}{\partial( x_{2} )^{2}} & \cdots & \frac{\partial^{2}x_{1}}{{\partial x_{2}}x_{n}} \\ \vdots & \vdots & \ddots & \vdots \\0 & \frac{\partial^{2}x_{1}}{{\partial x_{n}}x_{2}} & \cdots & \frac{\partial^{2}x_{1}}{\partial( x_{n} )^{2}}\end{bmatrix}$Note that there are n of such Hessian matrices, each having (n−1)²non-zero elements. The Hessian matrix of the measurements of a physicalsystem, satisfies Schwartz condition of continuous partialdifferentiability and is, therefore, symmetric. Hence, there aren(n−1)/2 distinct time-varying elements in each matrix which can beconsidered as global transit capturing features. Note that each Hessianmatrix can then be computed using numerical or automaticdifferentiation.

Some embodiments may utilize global transient capturing features of thelocal features. For example, a Jacobian of the local features vector ofone node with respect to another. Note that the global Hessian featureof the feature is computed as the second partial derivative of onefeature of a mentoring node, with respect to the vector field of thelocal features of another monitoring node. For instance, the Jacobian ofW¹ (local base features of monitoring node x₁) with respect to thevector field W² (local base features of monitoring node x₂) is a ƒ₁*ƒ₂matrix:

${J_{W^{2}}( W^{1} )} = \begin{bmatrix}\frac{\partial w_{1}^{1}}{\partial w_{1}^{2}} & \frac{\partial w_{1}^{1}}{\partial w_{2}^{2}} & \cdots & \frac{\partial w_{1}^{1}}{\partial w_{f_{2}}^{2}} \\\frac{\partial w_{2}^{1}}{\partial w_{1}^{2}} & \frac{\partial w_{2}^{1}}{\partial w_{2}^{2}} & \cdots & \frac{\partial w_{2}^{1}}{\partial w_{f_{2}}^{2}} \\ \vdots & \vdots & \ddots & \vdots \\\frac{\partial w_{f_{1}}^{1}}{\partial w_{1}^{2}} & \frac{\partial w_{f_{1}}^{1}}{\partial w_{2}^{2}} & \cdots & \frac{\partial w_{f_{1}}^{1}}{\partial w_{f_{2}}^{2}}\end{bmatrix}$

Some embodiments may use a Hessian of the features of one node withrespect to another. In this case, a global Hessian feature of thefeature may be computed as the second partial derivative of one featureof a mentoring node with respect to the vector field of the localfeatures of another monitoring node. For instance, the Jacobian of W¹(local base features of monitoring node x₁) with respect to the vectorfield W² (local base features of monitoring node x₂):H _(W) ₂ (W ¹)={H _(W) ₂ (w ₁ ¹),H _(W) ₂ (w ₁ ²), . . . ,H _(W) ₂ (w_(ƒ) ₁ ¹)}where H_(W) ₂ (w₁ ¹) is the Hessian of w₁ ¹ (first local feature of nodex₁) with respect to the vector field W² (local base features ofmonitoring node x₂). Having n monitoring nodes, the global Hessianfeature of features will be the elements of n such tensors (each beingan array of matrices).

Note that extracting transient capturing features involvesdifferentiations which is known to be a noise-prone process. In order tode-noise the resulting features, these features may be smoothened at thelocal and/or global levels. The smoothing filters might comprise, forexample, band-pass filters with very a low cut-in frequency (just tofilter the fictitious DC value which may exists as an artifact ofnumerical derivatives) and a cut-off frequency which may be selected byinspecting the power spectral density of the signals. The bandwidth ofthe filter can be automatically selected to be, for example, the first 5harmonics of the signal. The order of the filter may be selected by themaximum phase distortion allowed while still capturing the fastestnormal transient of the environment. For example, a noise-robust 5-pointdifferentiator can be derived as:

${\frac{dx}{dt}(t)} = \frac{{5{x(t)}} + {2{x( {t - T_{s}} )}} - {8{x( {t - {2T_{s}}} )}} - {2{x( {t - {3T_{s}}} )}} + {3{x( {t - {4T_{s}}} )}}}{8T_{s}}$Polynomial-based smooth noise-robust filters, such as a Savitzky-Golaysmooth differentiation filter, can also be used. According to someembodiments, FFT-based filters with zero-phase distortion can be appliedto smoothen transient capturing features. FFT-based filters maytransform the feature time series into the frequency domain and thenapply a zero-phase band-limited (or low-pass) filter. An inverse FFT canthen be performed to obtain a filtered feature time-series data.

During the off-line training phase, normal and abnormal data sets may begenerated (including fast transients), and the global feature vectors,including the smoothed transient capturing features, may be extracted totrain a classification decision boundary. The classification decisionboundary could be based on any classification method, such as SupportVector Machines (“SVM”), K-nearest neighborhood, deep learning neuralnetworks, Extreme Learning Machines (“ELM”), etc. The computed decisionboundary can then be pre-stored in the system for real-time operations.It might comprise, for example, a mathematical score function of theglobal features and a score threshold which determines whether thecurrent system status is normal or abnormal.

FIG. 13 illustrates an anti-jitter process 1300 in accordance with someembodiments. Note that during the real-time operations theclassification decision boundary computes a score at each sampling time,which is then compared to a pre-stored score threshold. In order toavoid jittering around the score threshold, which could happen due tothe fast-changing nature of the transient capturing features, ananti-jittering hysteresis mechanism may be applied around the threshold.Suppose that the decision boundary is trained such that a negative scoreresembles “normal” status and a positive score resembles “abnormal”status. Note that a hysteresis function is not necessarily symmetric andcan have two different trigger edges a>0 and b<0 as the score threshold(illustrated in FIG. 13).

The system may use the current system status to apply the proper scorethreshold. If the current status is “normal,” it remains “normal” untilscore>a, at which point the new status becomes “abnormal.” If thecurrent status is “abnormal,” it remains “abnormal” until score<b, atwhich point the new status becomes “normal.”

The family of transient capturing featuring described here constitutes alarge number of features that could be extracted in addition to the basefeatures to create a reliable decision boundary. Any subset of thesefeatures, or features of the similar nature, could be used for anyparticular application. Suppose there are n monitoring nodes, eachhaving f_(i), i=1, . . . , n local base features. The features of thetime-domain node values might include the first derivative, the secondderivative, Jacobian information (e.g., determinants, trace,eigenvalues, singular values), Hessian information, etc. The features ofthe base (local) features might include the first derivative, the secondderivative, Jacobian information (e.g., determinants, trace,eigenvalues, singular values), Hessian information, etc.

Current anomaly/threat detection methods do not perform well undersystem transients and misclassify the transient normal operation asabnormal, causing false alarms during transients. Embodiments describedherein may provide a reliable, low-cost, and computationally efficientsolution. Some advantages of embodiments described herein include:carrying out cyberattack detection during continuous operational mode(especially during rapid transients such as Dry Low NOx (“DLN”) modetransfer in a gas turbine); detection may be more sensitive to fasttransients and reduces false positive rate; an analytics application fordigital monitoring system, etc.

Embodiments might be tested using various simulations of a gas turbine.For example, an asset may have 20 monitoring nodes, each having 5 localbase features, including, median, standard deviation, kurtosis, range,and a moving average. The features may be extracted, for example, over asliding window of batch data of node measurements of size 50 second,sliding by one sampling time (Ts=1 sec) at each sampling time. Inaddition, one transient capturing feature might be added, namely thefirst derivative of the time-domain node values (rate features) as the6-th local feature for each node. The transient capturing feature foreach node might then pass through a 5-degree smoothing filter. At theglobal level, there may be two interactive features as the correlationof two monitoring nodes used in those features. The global featurevector might be comprised of 122 features (6 local per node plus 2global interactive). Then a classification decision boundary could betrained based on ELM neural networks using “normal” and “abnormal” datasets collected by simulating a high-fidelity model of the asset. The“normal” data set might be created by Pseudo-Random Binary Sequence(“PRBS”) excitation to resemble different operational conditions, andthe “abnormal” data set might be created by DoE. The features can thenbe extracted over a sliding window of the time-series data. The trainingdata set might comprise, for example, over 2 million data points, eachbeing a vector of size 122, in the feature space. The ELM training codeis implemented efficiently, using sparse matrix manipulations, to beable to handle the big data. To resemble real operations, thesimulations may be done in close-loop with the gas turbine controller inthe loop. The results may then be compared with the results of anotherclassification decision boundary (using the same classification methodsand same data sets) in which only base features are used (no transientcapturing features included) as the base-line. The performance of thereliable cyber-threat detection system in various test scenarios may notcreate false alarms during rapid normal transients and DLN modetransfers, while still detecting attacks even faster than the base-lineclassifier. The transient capturing features may improve bothsensitivity and accuracy of the detection system. Moreover, the reliablesystem may be computationally low-cost and not add a noticeable demandto real-time computational needs.

FIG. 14 is a more detailed architecture of a system 1400 to protect anindustrial asset according to some embodiments. The system 1400 extractslocal features for monitoring nodes 1-N 1420, 1422 from time-domainvalues 1410. The system 1400 then smooths local transient capturingfeatures for each node 1430, 1432. The system 1400 may also, accordingto some embodiments, extract global features 1440, smooth the globaltransient capturing features 1450, and compute a decision boundary score1460. According to some embodiments, the extraction of global features1440 may also utilize the time-domain values of monitoring nodes 1410(e.g., to allow cross-correlation as illustrated by the dashed arrow inFIG. 14). The decision boundary score can then be compared 1470 to athreshold value 1470 based on an applied threshold hysteresis 1480 and acurrent system status 1490 to determine if the new system status of theindustrial asset is “attack” or “normal.”

FIG. 15 is a method that might be associated with an on-line operationalprocess in accordance with some embodiments. After observing themonitoring nodes at S1510, the features are extracted at S1520 from eachobservation of each monitoring node. Then using the dynamic modelsidentified in a training phase, each model then generates filtered orestimated features at S1530 using stochastic estimation techniques, suchas Kalman filtering. In some embodiments, dynamic models may not berequired to further filter or estimate features. The covariance matrixof the process noise needed for the stochastic estimator is readilyavailable here as Q, which can be computed during training phase as thecovariance of the error term e(t). Then the output of each stochasticestimator is compared against its corresponding local decision boundaryat S1540, also computed and pre-stored during the training phase. If thelocal boundary is not passed at S1540, the monitoring node is normal atS1550. Each monitoring node with an estimated feature that violates thecorresponding decision boundary is reported as being under attack atS1560.

In the next stage, the system post-processes the localized attack anddetermines whether the detected attack is an independent attack or it isan artifact of the previous attack through propagation of the effects inthe closed-loop feedback control system at S1570. This may provideadditional information and insight and may be useful when multipleattacks are detected at the same time.

For example, FIG. 16 is a method of determining whether an attack is anindependent attack or a dependent attack according to some embodiments.According to some embodiments, three tests may be performed to determineif an attack should be classified as an “independent attack” or a“dependent attack:” (1) a causal dependency test, (2) a propagation pathtest, and (3) a time separation test. Together, these three tests arereferred to herein as the “attack dependency conformance test.” AtS1610, a causal dependency matrix may be used to determine if thecurrent attack was potentially caused by a previous attack. If thecurrent attack could not have been caused by a previous attack at S1610,it is classified as an “independent attack” at S1620. In this causalitytest, the system may check whether there is a potential causaldependency between the newly detected attack and any previously detectedattack on other monitoring nodes. This check might be based on, forexample, a binary matrix of causal dependencies between any two nodes(e.g., as described with respect to FIG. 16). The causal dependencymatrix might be generated; according to some embodiments, based ondomain knowledge. If no such possible dependencies exist, the attack isreported as an “independent attack” at S1620. Otherwise, the system mayperform a second check.

In particular, at S1630 a propagation paths map may be used to determineif the current attack potentially propagated from a previous attack. Ifthe current attack could not have propagated from a previous attack atS1630, it is classified as an “independent attack” at S1620. In thispropagation test, for each causal dependency the system may checkwhether a propagation path is fulfilled. This might mean that, forexample, if the effect of node 1 being under attack is propagated tonode 4, through node 3, then an anomaly in node 1 can cause an anomalyon node 4 only if node 3 is already anomalous. The anomaly propagationpaths might also be defined by domain knowledge and pre-stored in thelocalization system. If no such propagation paths are fulfilled, thenthe attack is reported an “independent attack” at S1620. Otherwise, thesystem may perform the third check.

At S1640, control loops time constraints may be used to determine if thecurrent attack was potentially caused by a previous attack based on timeseparation. If the current attack could not have been caused by aprevious attack based on time separation at S1640, it is classified asan “independent attack” at S1620. This time separation test may utilizethe fact that if the attacked monitoring under investigation is anartifact of the closed-loop feedback system, then the effect shouldarise within a time window between the rise time and the settling timeof the control loop corresponding to the monitoring node. However, sincethe system uses a dynamic estimator, a propagation time may need to beadded throughout the estimator. Using n features, and p lags in themodels, the dynamic estimator will have n*p states, and therefore addsn*p sampling times delay into the system. Therefore, the expected timewindow for a dependent attack to occur might be defined by:1.5*τ+n*p<Δt<5*τ+n*pwhere Δt is the time after any previously detected attacks on othernodes that has passed checks 1 and 2, and τ is the time constant of thecontrol loop responsible for the current node under investigation. Ifsuch a time-separation check is not passed, the system reports theattack as an independent attack at S1620.

If it is determined at S1650 that the current attack meets the timeseparation test (and, therefore, also meets both the propagation test ofS1630 and the causal dependency test of S1640), the current attack isclassified as a “dependent attack” at S2150.

Note that other attack and anomaly detection techniques may only providea binary status of the overall system (whether it is under attack ornot). Embodiments described herein may provide an additional layer ofinformation by localizing the attack and determining not only if thesystem is under attack (or not) but also which node is exactly underattack.

As a result, embodiments may provide a significant and automatedsolution to attack localization. Note that the attack localizationinformation may be important when responding to the attack, includingoperator action plans and resilient control under attack. Embodimentsdescribed herein may handle multiple simultaneous anomalies in thesystem, which is beyond the capability of the conventional faultdetection systems. This may also let the approaches described herein beused as a fault detection and isolation technique for moresophisticated, multiple-fault scenarios. Further, distributed detectionand localization systems enabled by embodiments described herein acrossmultiple equipment and systems may allow for a coordination of data todetect and precisely pin-point coordinated multi-prong attacks. This mayfurther enable a relatively quick way to perform forensics and/oranalysis after an attack.

Note that some embodiments may analyze information in the feature space,which has many advantages over working in the original signal spaces,including high-level data abstraction and modeling high dimensionalspaces without adding substantial computational complexity. Thefeature-based method for localization may also extend feature vectorsand/or incorporate new features into existing vectors as new learningsor alternate sources of data become available. Embodiments describedherein may also enable use of heterogeneous sensor data in a large-scaleinterconnected system, even when the data comes from many geospatiallylocated heterogeneous sensors (i.e., conventional plant sensors,unconventional sensors such as cell-phone data, logical, etc.). This mayoffer additional commercial advantages for post-mortem analysis after anattack.

FIG. 17 illustrates a feature time series 1700 of a first attack examplecomparing the real-time feature of a monitoring node to the modeledfeature of a monitoring node via a graph 1710 according to someembodiments. In particular, the examples described with respect to FIGS.17 through 16 involve the following parameters for a gas power turbine(similar to those values described with respect to FIGS. 4 through 6):

-   -   Compressor Discharge Pressure (“CPD”),    -   Compressor Discharge Temperature (“CTD”),    -   Compressor Inlet Temperature (“CTIM”),    -   Turbine Fuel Flow (“FQG”),    -   Generator Electrical Power Output (“DWATT”), and    -   Turbine Exhaust Temperature (“TTXM”).        Consider, for example, an attack on TTXM. In this single attack        scenario, the system may want to verify whether it can detect        and localize the attacked node. As illustrated in FIG. 17, the        attack is detected at t=11 sec. Using the embodiments described        herein, the attack is detected within 1 sec and correctly        localized to TTXM. FIG. 17 shows the measured feature time        series of the detected and localized attack 1730 along with the        generated features 1720 estimated using stochastic model-based        estimation.

FIG. 18 illustrates a feature time series 1800 via a graph 1810 of asecond (stealthy) attack comparing the real-time feature of a monitoringode to the modeled feature of a monitoring node in accordance with someembodiments. That is, this is again an attack on TTXM but this time theattack simulates a stealthy attack in which the sensor is tampered withslowly over time and/or elaborately. Such stealthy attacks are designedto pass the existing fault diagnosis system and can remain in thecontrol system for a long time without being detected. In thissimulation, the attack was applied at t=40 sec. Using the localizationmethods described herein, the attack was detected at t=105 sec, and iscorrectly localized to TTXM. FIG. 18 shows the measured feature timeseries of the detected and localized attack 1830 along with the expectedfeatures 1820 estimated using the stochastic model-based estimation.

In a third attack scenario, the system may simulate a simultaneousattack on two monitoring nodes. Two sensors are attacked at the sametime, namely CPD and CTD, and both attacks are applied at t=15 sec.Using embodiments described herein, both attacks are truly detected andlocalized within seconds. Out of the other 4 sensors, 3 are correctlynot detected at all. One is detected (DWATT) at a later time, which isdependent attack. The results are summarized in the table 1900 of FIG.19. In particular, the table 1900 lists the attack nodes 1902 along withassociated externally attacked data 1904 and attack detection andlocalization data 1906.

In this third example (illustrated in the table 1900), there are twoexternally injected attacks on CPD and CTD. The first attack is detectedat t=16 sec and localized to CTD. Since there is no previously detectedattack, the causality test fails and this attack is correctly reportedas an “independent attack.” The second attack is detected at t=19 secand correctly localized to CPD. In this case, there is causal dependencyand a direct proportion path from CTD to CPD. The causal dependencymatrix 2000 for this example is shown in FIG. 20. The matrix 2000 listseach potential attack node and whether or not that node can have aneffect on each other node (with a “1” indicating a potential effect anda “0” indicating no potential effect).

The second attack therefore passes both the causality test and theproportion test. However, based on time separation criterion, in orderfor the CPD attack to be a dependent attack it must have happened within4.25<Δt<9.5 sec after the CTD detection instance. The actual Δtillustrated in the table 1900 is 3 sec (that is, 19 sec−16 sec).Therefore, the time separation test is not passed and, as a result, theCPD attack is correctly reported as an “independent attack.”

At t=53 sec, the DWATT sensor is also reported as being under attack.Note that there are two previously reported attacks, and the causalityand propagation tests pass for both previous attacks (as shown in thematrix 2000). Using the time separation criterion, the DWATT attackinstant must be with 15.5<Δt<47 sec after those attacks. The table 1900lists the actual Δt as Δt=53 sec−16 sec=37 sec for CTD attack and Δt=53sec−19 sec=34 sec for CPD attack. So, the time separation test passesfor both previous attacks and, therefore, the DWATT attack is correctlyreported as a “dependent attack.” Note that, based some embodimentsdescribed herein, passing the time separation test even for onepreviously detected attack may still be enough to report DWATT as adependent attack.

FIG. 21 shows an architecture for an autonomous reconfigurable virtualsensing system 2100. The system 2100 receives time-series measurements2120 of the sensors as inputs. The measurements are pre-filtered 2120for de-noising and outlier removal. Denoising may be done, for example,by low pass filtering using law pass filters whose individual cut-offfrequencies may be turned based on the individual bandwidths of eachsensor. Outlier removal might be performed online by computing thestandard deviation of measurements over a sliding window. For example,FIG. 22 illustrates a sliding window 2200 including a series of valuesper second. Referring again to FIG. 21, feature extraction 2140, anomalydetection 2150, and localization techniques 2154 may be used todetermine 2152 if there is any anomaly in the sensor (and to specify theparticular anomalies). When an anomaly or abnormality exists in thesystem 2100, all the sensor measurements may be passed, via an indexedselector 2130, to an autonomous, resilient estimator 2180 that uses anonline continuous learning technique in accordance with any of theembodiments described herein. When there is an abnormality, the sensorsthat are determined by conformance matrix logic 2160 as an independentanomaly (i.e., and not an artifact of the propagation of other anomaliesthrough the system 2100) are removed 2170 and the uncomplimented subsetof sensors are passed to the autonomous, resilient estimator 2180. Forexample, the system 2100 may have N sensors, of which p sensors arenormal and q sensors are independently abnormal. Note that both p and qare time-varying but p[k]+q[k]=N at each time instant k. The p normalsensors are specified by the conformance matrix logic 2160 anddown-selected via the indexed selector 2130 to be inputted to theautonomous, resilient estimator 2180. Note that the normal subset may becontinuously changing and, as a result, the internal learningconfiguration of the autonomous, resilient estimator 2180 is alsochanging. The online continuous learning is used to learn a(potentially) nonlinear, time-varying, and variable-structure function ƒthat relates the next-step values of the sensors estimates to thecurrent and lagged values of the sensor estimates (i.e., outputs of theautonomous, resilient estimator) and the current and lagged values ofthe normal sensor measurement (i.e., inputs of the autonomous, resilientestimator) as follows:Ŝ=[Ŝ ₁ Ŝ ₂ . . . Ŝ _(N)]^(T)Ŝ ^(n)=[Ŝ ₁ ^(n) Ŝ ₂ ^(n) . . . Ŝ _(p) ^(n)]^(T) ,Ŝ ^(a)=[Ŝ ₁ ^(a) Ŝ ₂^(a) . . . Ŝ _(q) ^(a)]^(T)Ŝ[K+1]=ƒ(Ŝ[k], . . . ,Ŝ[k−l],Ŝ ^(n)[k], . . . ,Ŝ ^(n)[k−m],k)where l and m are the number of lags used for outputs and inputs,respectively; and the normal and abnormal sensors are depicted withsuperscripts, n and a, respectively. Note that both l and m might alsobe found automatically online and they might be time varying as well,hence making ƒ a variable structure. For substantially large-scalesystems, a sparsity structure might be exploited in the autonomous,resilient estimator 2180 to have a reduced-order observer, or to have afull order observer in which continuous learning computations mayapplied at each configuration change event until convergence isachieved. Estimator parameters may then remain constant until the nextconfiguration change occurs.

The embodiments described herein may be implemented using any number ofdifferent hardware configurations. For example, FIG. 23 is a blockdiagram of an industrial asset protection platform 2300 that may be, forexample, associated with the systems 100, 300, 800, 1000 of FIGS. 1, 3,and 10A respectively. The industrial asset protection platform 2300comprises a processor 2310, such as one or more commercially availableCentral Processing Units (“CPUs”) in the form of one-chipmicroprocessors, coupled to a communication device 2320 configured tocommunicate via a communication network (not shown in FIG. 23). Thecommunication device 2320 may be used to communicate, for example, withone or more remote monitoring nodes, user platforms, digital twins, etc.The industrial asset protection platform 2300 further includes an inputdevice 2340 (e.g., a computer mouse and/or keyboard to input virtualsensor and/or predictive modeling information) and/an output device 2350(e.g., a computer monitor to render a display, provide alerts, transmitrecommendations, and/or create reports). According to some embodiments,a mobile device, monitoring physical system, and/or PC may be used toexchange information with the industrial asset protection platform 2300.

The processor 2310 also communicates with a storage device 2330. Thestorage device 2330 may comprise any appropriate information storagedevice, including combinations of magnetic storage devices (e.g., a harddisk drive), optical storage devices, mobile telephones, and/orsemiconductor memory devices. The storage device 2330 stores a program2312 and/or a virtual sensor model 2314 for controlling the processor2310. The processor 2310 performs instructions of the programs 2312,2314, and thereby operates in accordance with any of the embodimentsdescribed herein. For example, the processor 2310 may determine that atleast one abnormal monitoring node is currently being attacked orexperiencing a fault. The processor 2310 may also continuously executean adaptive learning process to create or update virtual sensor modelsfor the monitoring nodes. Responsive to an indication that a monitoringnode is currently being attacked or experiencing a fault, the processor2310 may be dynamically reconfigured to estimate a series of virtualnode values. for the abnormal monitoring node or nodes based oninformation from normal monitoring nodes and appropriate virtual sensormodels. The series of monitoring node values from the abnormalmonitoring node or nodes may then be replaced with the virtual nodevalues

The programs 2312, 2314 may be stored in a compressed, uncompiled and/orencrypted format. The programs 2312, 2314 may furthermore include otherprogram elements, such as an operating system, clipboard application, adatabase management system, and/or device drivers used by the processor2310 to interface with peripheral devices.

As used herein, information may be “received” by or “transmitted” to,for example: (i) the industrial asset protection platform 2300 fromanother device; or (ii) a software application or module within theindustrial asset protection platform 2300 from another softwareapplication, module, or any other source.

In some embodiments (such as the one shown in FIG. 23), the storagedevice 2330 further stores a virtual sensor database 2400. An example ofa database that may be used in connection with the industrial assetprotection platform 2300 will now be described in detail with respect toFIG. 24. Note that the database described herein is only one example,and additional and/or different information may be stored therein.Moreover, various databases might be split or combined in accordancewith any of the embodiments described herein.

Referring to FIG. 24, a table is shown that represents the virtualsensor database 2400 that may be stored at the industrial assetprotection platform 2300 according to some embodiments. The table mayinclude, for example, entries identifying industrial assets to beprotected. The table may also define fields 2402, 2404, 2406, 2408,2410, 2412, 2414 for each of the entries. The fields 2402, 2404, 2406,2408, 2410, 2412, 2414 may, according to some embodiments, specify: anindustrial asset identifier 2402, an industrial asset description 2404,a virtual sensor identifier 2406, a matrix 2408, description 2410, astatus 2412, and a neutralization level 2414. The virtual sensordatabase 2400 may be created and updated, for example, when a newphysical system is monitored or modeled, an attack is detected, etc.

The industrial asset identifier 2402 and description 2404 may define aparticular machine or system that will be protected. The virtual sensoridentifier 2406 might be a unique alphanumeric code identifying aparticular sensor being modeled for the industrial asset. The matrix2408 might be associated with a correlation heat map or lookup table,the description 2410 might indicate what sensor is being estimated, andthe status 2412 might indicate, for example, whether the associatedmonitoring node is operating normally or is currently undergoing acyber-attack, experience a fault, and/or is being replaced (e.g., with a“predicted” value”). The neutralization level 2414 may be based, forexample, on confidence levels and/or the importance of a sensor beingattacked (or experiencing a fault). FIG. 25 is an example of a virtualsensor display 2500 that might be used, for example, to provideinformation 2510 to an operator and/or to provide an interactiveinterface allowing an operator to adjust virtual sensors as appropriate.Selection of an element on the display 2500 (e.g., via a touchscreen)might, for example, result in the presentation of more information aboutthat element (e.g., via a popup window), allow an operator to adjustparameters associated with the element, etc.

FIG. 26 shows a system 2600 that uses an autonomous, resilient estimator2670 in a controls and analytics platform. In particular, sensormeasurement time-series values are combined 2640 with plant set-pointsand the result goes to a switch with bumpless transfer control 2630 viaa controller 2610 and a plant 2620. The sensors measurements time-seriesvalues also under go pre-filtering 2650 before being passed to theautonomous, resilient estimator via a first indexed selector 2660. Asecond indexed selector 2680, controlled by sensor diagnostics andanomaly classification 2690 receives data from the autonomous, resilientestimator 2670 and provides information for sensor software redundancy,sensor health analysis, and control of the switch 2630.

The healthy estimates of the abnormal sensors and their indices are theprovided into the control loop and are used to replace of the originalabnormal measurements. This is done through the switch with bumplesstransfer control 2630 that might utilize any bumpless switchingmechanism (such as a bumpless Proportional-Integral-Derivative (“PID”),a switched adaptive controller, a smooth transition controller, etc.).During normal operation, the switch 2630 is open and thus the plantsensor measurements are passed through the feedback loop.

When an anomaly is detected, the switch 2630 is closed and the virtualhealthy estimated of the abnormal sensors are passed to the controlfeedback loop. The bumpless transfer control may help ensure smoothnessof the signals during a transition and avoids abrupt (and potentiallydestabilizing) spikes in the control loop. The sensor measurementtime-series may be a combination of the virtual sensor estimates(replacing the independently compromised sensors) and the original plantsensors that are not independently compromised. This mechanism may helpneutralize the effect of the abnormal measurements (which could be duethe abnormality of the sensor itself, such as a sensor fault, or acyber-attack on the sensor) and maintains healthy operations of theplant. Note that the switch 2630 can be re-opened as soon as the plantstatus is back to normal (again with bumpless transfer control) or mayremain latched in for some additional period time and opened after thatdelay. According to some embodiments, the estimates of the abnormalmeasurement are also used for further health analytics. The system 2600may also produce estimates of healthy measurements in real-time. Theseestimates may remain in “stand-by” and when any of those sensors becomesabnormal the 2600 system can adopt a new configuration. According tosome embodiments, these estimates also provide software redundancy toincrease the reliability of plant operations.

Some embodiments described herein may provide systems and/or methods forautonomous reconfigurable virtual sensing to neutralize the effect ofanomalies (e.g., cyber-attack or faults) in system measurements.Embodiments may provide correct estimates of compromised sensormeasurements using uncompromised sensor measurements, thus replacing thecomprised sensors with healthy virtual (or “soft”) sensors. According tosome embodiments, an autonomous, resilient estimator may use continuousadaptive learning. The virtual sensor estimations may be computed onlineusing an adaptive recursive method, such as one based on RL. Embodimentsmay be scalable and efficient and automatically adjust a configurationto accommodate a time-varying uncompromised portion of system sensors.Moreover, embodiments may work with partial or no a priori knowledge(e.g., a model).

Some embodiments may use conformal prediction to automatically determinewhether neutralization is feasible for a current system operationalregion and/or when a healthy sensor or a safe shut down is appropriate.Also, once the neutralization is complete, some embodiments utilize ahuman-in-the-loop decision mechanism to determine if/when the physicalsensors can be brought back into the control loop (e.g., as described inconnection with FIG. 28).

FIG. 27 is an adaptive, self-tuning neutralization system 2700 accordingto some embodiments. As before, the system 2700 uses an autonomous,resilient estimator 2770 in a controls and analytics platform. Sensormeasurement time-series values are combined 2740 with plant set-pointsand the result goes to a switch with bumpless transfer control 2730 viaa controller 2710 and a plant 2720. The sensors measurements time-seriesvalues are also pre-filtered 2750 before being passed to the autonomous,reconfigurable resilient estimator 2770 via a first indexed selector2760. A second indexed selector 2780, controlled by sensor diagnosticsand anomaly classification 2790 receives data from the autonomous,resilient estimator 2770 and provides information for sensor softwareredundancy, sensor health analysis, and control of the switch 2730.According to some embodiments, confidence scores may be provided fromthe sensor diagnostics, anomaly classification aid localization 2790 tothe autonomous, reconfigurable resilient estimator 2770 and/or from theautonomous, reconfigurable resilient estimator 2770 to the switch 2730.

Together, the controller 2710, plant 2720, and switch 2730 may et as a“control loop” for the system 2700. The healthy estimates of theabnormal sensors and their indices are the provided into the controlloop and are used to replace of the original abnormal measurements. Thisis done through the switch with bumpless transfer control 2730 thatmight utilize any bumpless switching mechanism (such as a bumpless PID),a switched adaptive controller, a smooth transition controller, etc.).During normal operations, the switch 2730 is open and thus the plantsensor measurements are passed through the feedback loop. When ananomaly is detected, the switch 2730 is closed and the virtual healthyestimates of the abnormal sensors are passed to the control feedbackloop. The bumpless transfer control may help ensure smoothness of thesignals transition an avoid abrupt (and potentially destabilizing)spikes in the control loop. The sensor measurement time-series maycomprise, for example a combination of the virtual sensor estimates(replacing the independently compromised sensors) and the original plantsensors, which are not independently compromised. This may helpneutralize the effect of the abnormal measurements (which could be duethe abnormality of the sensor itself, that is a fault or a cyber-attackon the sensor) and help maintain healthy operation of the asset. Theswitch 2730 may be opened as soon as the asset status returns to normal(again with bumpless transfer control) or may remain latched for someadditional time (and be opened with after the time delay). The estimatesof the abnormal measurement may also be used for further healthanalytics. According to some embodiments, the system 2700 may produceestimates of the healthy measurements in substantially real-time. Theseestimates may remain in stand-by mode until any of those sensors becomesabnormal (and, as a result, the system 2700 adopts a new configuration).These estimates may also serve as a software redundancy to increasereliability of asset operations.

Some embodiments may not only detect and localize cyber-attacks on aprotected asset but also allow for the continuing operation of the assetduring cyber-attacks or faults via “neutralization” (e.g., providingactive tunable resilience). The system neutralization might proceed atdifferent neutralization modes and performance levels of assetprotection based on, for example: an identified criticality of anencountered cyber-attack, an identified criticality of a protectedasset, a current operating mode of a protected asset, etc. According tosome embodiments, the system 2700 may analyze the attack and the statusof the asset in substantially real-time and decide on an appropriateneutralization performance level. Note that neutralization performancemay be highly dependent on the observability characteristics of the nodesuite (i.e., protected sensors and actuator set) of the protected assetthat may be a result of; for example: redundancy, physical location,and/or type of sensors/actuators. The observability of an attackedsubset of nodes may be determined in substantially real-time during anattack by the neutralization module that may determine theneutralization mode. For example, if the attacked subset of nodes ishighly observable from the non-attacked nodes, neutralization might useresilient virtual estimation of the attacked nodes and the goal ofneutralization might be continual operation of the asset with minimalperformance degradation. However, if the real-time observabilityanalysis determines that the attacked subset is not observable from theremaining non-attacked nodes, neutralization might aim for operationwith a degraded (but secure) performance. In some cases, the appropriatelevel of neutralization might even be associated with a safe shutdown ofthe asset. The real-time observability analysis may also enableestimation error statistics of the virtually sensed nodes. Thesestatistics might be used, for example, to determine and/or display theconfidence boundaries of the neutralization that is being performed.These confidence boundaries might, for example, be observed by plantoperators and allow them to take appropriate protective actions based onthe following:

-   -   a Failure Mode and Effect Analysis (“FMEA”), and/or    -   a Threat Mode and Effect Analysis (“TMEA”).

According to some embodiments, neutralization performance levels andconfidence boundaries may also depend on confidence levels of thedetection and localization of one or more cyber-attacks. For example, ifan attack is localized with a limited confidence then the confidenceboundaries of neutralization may be relatively large as compared to anattack that was localized with a higher level of confidence.

According to some embodiments, the autonomous, reconfigurable resilientestimator 2770 for sensor attack neutralization is based at least inpart on a conformal prediction. The conformal prediction may, forexample, use past experience to determine a level of confidence in acurrent decision/estimation (e.g., the “confidence scores” in FIG. 27).The autonomous, reconfigurable resilient estimator 2770 may beautomatically reconfigured based on the available healthy sensors (whichmay include a subset that are hardened and hence are always trustable)and estimated values for the rest of the sensors. The system 2700 mayalso calculate a confidence level for each sensor estimate (e.g., byusing POMDP and/or Q-learning). This information may be passed intoswitching control logic that decides whether to switch to the virtualsensor for each individual sensor. It may also decide overall, based onthe current system situation, whether or neutralization is feasible.Note that this switching control logic may have its own training (whichmight also include a predefined set of rules based on system regions ofasset operation, set-points, etc.).

According to some embodiments, the system 2700 might determine that asensor is no longer being attacked (or is no longer experiencing afault) and, as a result, it might be appropriate to replace virtualestimated values with the actual monitoring node values generatedsensor. FIG. 28 is logic 2800 for switching back from virtual sensors tophysical sensors in accordance with some embodiments. In particular, thelogic 2800. illustrates some factors that may be (e.g., afterneutralization is accomplished) considered when deciding whether or notswitch back the controller to physical sensors (and switch off virtualsensors). An and operation 2810 may receive an indication of whether avirtual sensing residual remained small, an indication of whether aglobal status has remained normal, and/or an external indication noactive intrusion into the network is being experience. Based on all ofthese factors, a human operator may be asked to attest 2820 that thevirtual sensor information should be replace with actual, physical data.If the operator does not do so, then the virtual sensors may be kept inthe loop 2830. If the operator does provide the attestation 2820, thenthe system may switch back to actual, physical data 2840.

According to some embodiments, the switch back 2840 may be “ordered”such that is performed in accordance with a sensor ranking list. Thatis, the sensors may be switched back one at a time according the sensorranking list, starting with the lowest ranked (least critical) sensor.There might be, in some embodiments, a minimum waiting time after eachsensor is switched back before the next sensor is considered. Thewaiting time might be determined using simulations and/or may beselected based on an amount of time it takes to neutralize the systemusing the virtual sensor in the loop (e.g., once the correspondingphysical sensor is attacked and all other physical sensors are healthy).

If the global status becomes abnormal, the switch back 2850 process maybe halted, and all the virtual sensors might be switched into the loop.After all physical sensor are switched back, if global status becomesabnormal within a certain time window (e.g., based on a maximumneutralization time of all virtual sensors in simulations) then all ofthe virtual sensors may again be switched in the control loop. In thiscase, the virtual sensors may be switched off only by a manual overridefrom an operator with sufficient administrator authority.

The network intrusion detection could be based on network monitoringusing supervised or unsupervised anomaly detection methods, signaturemethod detection or be based on network intrusion detection, such asQuantum Key Distribution (“QKD”). As used herein, the term “QKD” mayrefer to a communication method that implements a cryptographic protocolutilizing components of quantum mechanics. Such network intrusiondetection methods may help protect network integrity. Being an activedefense solution, some embodiments may automatically neutralize attackedsensors by replacing compromised sensors with virtual sensor estimates.Once the system is completely neutralized and disinfected, embodimentsmay switch the system back to use the original physical sensors througha logical mechanism, in which one of the checks is to make sure thenetwork is uncompromised. Furthermore, since QKD can provide reliableeavesdropping detection capabilities, it may also provide a valuableindication for detecting replay attacks without interfering withphysical system performance.

Thus, a virtual sensing system may get a portion of the sensormeasurements that are healthy and uncompromised and then use thatinformation to provide healthy estimations for the measurements ofsensors that are compromised. Moreover, embodiments may improve cybersecurity and accommodate critical functionality associated with anindustrial asset. Some embodiments may by-pass signals from attackedsensors using estimated signals created using data from healthy sensors.This approach may allow for a correction mechanism to sustain theoperations while alerting operators about a cyber-attack or fault. Insome embodiments, an adaptive, reconfigurable resilient estimation forsensor attack neutralization may get a portion of the sensormeasurements that are healthy and uncompromised and use that informationto provide healthy estimations for the measurements of the sensors thatare compromised.

Since the compromised/uncompromised portions of measurements can be anysubset of the system sensors, this is essentially a combinatorialproblem that requires that a substantial number of estimation models bedeveloped and stored (and hence development times may be substantial).Such a brute force approach may build a model as a look-up table ofparameters for each sensor using least squares regression (or otherparameter estimation techniques) offline and save all of the models inthe system. In addition to a large development effort, this approach mayrequire vast amount of memory and computational power. By providing aself-tuning resilience level, embodiments may automatically decide whento neutralize the sensor attacks (as opposed to requiring a safe shutdown).

Some technical advantages of the embodiments described herein include:replacing faulty/attacked sensors with corrected estimates to provide aresilient estimation for attack neutralization, eliminating some sensorsto reduce cost (e.g., in a gas turbine one could replace low and highshaft speed sensor, respectively, with virtual sensing); providing asurrogate back-up for critical and/or unreliable sensors; improvingcontrol performance by increasing the number of sensors that areavailable (including sensor that are difficult or expensive to directlymeasure), etc. Other advantages may include reduced asset down-timeresulting from to cyber-attack incents and faults, increased assetreliability (resulting from software and algorithmic redundancy),reduced sensor cost, etc.

The following illustrates various additional embodiments of theinvention. These do not constitute a definition of all possibleembodiments, and those skilled in the art will understand that thepresent invention is applicable to many other embodiments. Further,although the following embodiments are briefly described for clarity,those skilled in the art will understand how to make any changes, ifnecessary, to the above-described apparatus and methods to accommodatethese and other embodiments and applications.

Although specific hardware and data configurations have been describedherein, note that any number of other configurations may be provided inaccordance with embodiments of the present invention (e.g., some of theinformation associated with the databases described herein may becombined or stored in external systems). For example, although someembodiments are focused on gas turbine generators, any of theembodiments described herein could be applied to other types of assets,such as dams, the power grid, autonomous vehicles, military devices,etc.

According to some embodiments, a virtual sensor model may be created(e.g., via an adaptive learning process) to replace a correspondingsensor monitoring node when needed. According to other embodiments,similar approaches may be taken with respect to other types ofmonitoring nodes. For example, a virtual model might replace an actuatormonitoring node or a controller monitoring node that is currentlyexperiencing an abnormality.

The present invention has been described in terms of several embodimentssolely for the purpose of illustration. Persons skilled in the art willrecognize from this description that the invention is not limited to theembodiments described, but may be practiced with modifications andalterations limited only by the spirit and scope of the appended claims.

The invention claimed is:
 1. A system to protect an industrial asset,comprising: a plurality of monitoring nodes, each monitoring nodegenerating a series of monitoring node values over time that represent acurrent operation of the industrial asset; an abnormality detectioncomputer to determine that at least one abnormal monitoring node iscurrently being attacked or experiencing a fault; and an autonomous,resilient estimator, coupled to the plurality of monitoring nodes andthe abnormality detection computer and having an estimation errorstandard deviation substantially comparable to each monitoring node, to:(i) continuously execute an adaptive learning process to create orupdate virtual sensor models for the monitoring nodes, (ii) responsiveto an indication that the at least one abnormal monitoring node iscurrently being attacked or experiencing a fault, automaticallydetermine a level of neutralization, (iii) dynamically reconfigure theautonomous, resilient estimator to estimate a series of virtual nodevalues for the abnormal monitoring node or nodes based on informationfrom normal monitoring nodes, appropriate virtual sensor models, and thedetermined level of neutralization, and (iv) replace the series ofmonitoring node values from the abnormal monitoring node or nodes withthe series of virtual node values.
 2. The system of claim 1, wherein thedetermined level of neutralization is based on: (i) an identifiedcriticality of an encountered cyber-attack, (ii) an identifiedcriticality of the industrial asset, and (iii) a current operating modeof the industrial asset.
 3. The system of claim 1, wherein thedetermined level of neutralization is associated with at least one of:(i) minimal performance degradation, (ii) degraded but secureperformance, and (iii) a safe shutdown procedure.
 4. The system of claim1, wherein the determined level of neutralization is associated withestimation error statistics of a virtually sensed node.
 5. The system ofclaim 4, wherein the estimation error statistics are associated withconfidence boundaries for at least one of: (i) failure mode and effectanalysis, and (ii) threat mode and effect analysis.
 6. The system ofclaim 1, wherein the adaptive learning process is associated with areinforcement learning method.
 7. The system of claim 6, wherein thereinforcement learning method is associated with at least one of: (i)Q-learning, (ii) a recursive least-squares method, and (iii) a recursiveweighted least-squares method.
 8. The system of claim 1, wherein theautonomous, resilient estimator is associated with a partially observedMarkov decision process with continuous state and action spaces.
 9. Thesystem of claim 1, wherein the virtual sensor models are created orupdated based on at least one of: (i) an analysis of variance, and (ii)a correlation/regression analysis.
 10. The system of claim 1, whereinthe dynamic reconfiguration is associated with at least one of: (i) anindexed selector, (ii) bumpless transfer control, and (iii) aproportional-integral-derivative controller, (iv) a switched adaptivecontrol, and (v) a smooth transition controller.
 11. The system of claim1, wherein the autonomous, resilient estimator estimates the series ofvirtual node values directly in time space after pre-filtering.
 12. Thesystem of claim 11, wherein the pre-filtering includes: (i) de-noisingvia low pass filtering using low pass filters with individual cut-offfrequencies tuned based on individual bandwidths of each monitoringnode, and (ii) outlier removal by computing a standard deviation ofmeasurement over a sliding window.
 13. The system of claim 1, whereinthe autonomous, resilient estimator further: (i) is unbiased with azero-mean error, (ii) has white Gaussian noise associated with aCramer-Rao information bound, and (iii) is statistically efficient withan error asymptotically converging to zero.
 14. The system of claim 1,wherein the industrial asset is associated with at least one of: (i) aturbine, (ii) a gas turbine, (iii) a wind turbine, (iv) an engine, (v) ajet engine, (vi) a locomotive engine, (vii) a refinery, (viii) a powergrid, (ix) a dam, and (x) an autonomous vehicle.
 15. A computerizedmethod to protect an industrial asset associated with a plurality ofmonitoring nodes, each monitoring node generating a series of monitoringnode values over time that represent current operation of the industrialasset, comprising: determining, by an abnormality detection computer,that at least one abnormal monitoring node is currently being attackedor experiencing a fault; continuously executing, by an autonomous,resilient estimator, an adaptive learning process to create or updatevirtual sensor models for the monitoring nodes; responsive to anindication that the at least one abnormal monitoring node is currentlybeing attacked or experiencing a fault, automatically determining alevel of neutralization, wherein the determined level of neutralizationis based on a confidence boundary associated with a virtually sensednode; and dynamically reconfiguring the autonomous, resilient estimatorto estimate a series of virtual node values for the abnormal monitoringnode or nodes based on information from normal monitoring nodes,appropriate virtual sensor models, and the determined level ofneutralization; and replacing the series of monitoring node values fromthe abnormal monitoring node or nodes with the series of virtual nodevalues.
 16. The method of claim 15, wherein the determined level ofneutralization is based at least in part on at least one of: (i) anidentified criticality of an encountered cyber-attack, (ii) anidentified criticality of the industrial asset, and (iii) a currentoperating mode of the industrial asset.
 17. The method of claim 15,wherein the determined level of neutralization is associated with atleast one of: (i) minimal performance degradation, (ii) degraded butsecure performance, and (iii) a safe shutdown procedure.
 18. The methodof claim 15, wherein the determined level of neutralization isassociated with estimation error statistics of a virtually sensed node.19. The method of claim 18, wherein the estimation error statistics areassociated with confidence boundaries for at least one of: (i) failuremode and effect analysis, and (ii) threat mode and effect analysis. 20.The method of claim 15, wherein the adaptive learning process isassociated with a reinforcement learning method.
 21. A non-transitory,computer-readable medium storing instructions that, when executed by acomputer processor, cause the computer processor to perform a method toprotect an industrial asset associated with a plurality of monitoringnodes, each monitoring node generating a series of monitoring nodevalues over time that represent current operation of the industrialasset, the method comprising: determining, by an abnormality detectioncomputer, that at least one abnormal monitoring node is currently beingattacked or experiencing a fault; continuously executing, by anautonomous, resilient estimator, an adaptive learning process to createor update virtual sensor models for the monitoring nodes; responsive toan indication that the at least one abnormal monitoring node iscurrently being attacked or experiencing a fault, dynamicallyreconfiguring the autonomous, resilient estimator to estimate a seriesof virtual node values for the abnormal monitoring node or nodes basedon information from normal monitoring nodes and appropriate virtualsensor models; replacing the series of monitoring node values from theabnormal monitoring node or nodes with the series of virtual nodevalues; and responsive to an indication that the at least one abnormalmonitoring node is no longer currently being attacked or experiencing afault, replacing the series of virtual node values with the series ofmonitoring node values from the abnormal monitoring node is no longercurrently being attacked or experiencing a fault.